Image pickup system

ABSTRACT

In the image pickup system of the present invention, factors that have an effect on noise such as the signal level, temperature of the CCD during shooting, exposure time, gain and the like are dynamically acquired, the noise level of the CCD is estimated, for example, for each pixel by a noise estimating unit, and signal components equal to or less than this noise level in the video signals are suppressed by a noise reducing unit, so that a high-quality image that is substantially free of noise is obtained while preserving the edges of the image and the like.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of U.S. patent application Ser. No.10/630,438, filed Jul. 30, 2003, which claims priority to JapaneseApplication No. 2002-229059 filed in Japan on Aug. 6, 2002, which areincorporated by reference as if fully set forth.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup system and imageprocessing program which reduce random noise arising in the image pickupelement system.

2. Description of the Related Art

Generally, noise components are contained in digitized signals obtainedfrom image pickup elements and the associated analog circuits and A/Dconverters. Such noise components can be divided into two maincategories, i.e., fixed pattern noise and random noise.

The abovementioned fixed pattern noise is noise that originates mainlyin the image pickup elements, as typified by defective pixels or thelike.

On the other hand, random noise is generated in the image pickupelements and analog circuits, and has characteristics that are close towhite noise characteristics.

In regard to the latter random noise, for example, a technique in whichthe amount of noise N is converted into a function by N=ab^(cD) usingconstant terms a, b and c that are statically given as constant terms,and the signal level D converted into a density value, the amount ofnoise N for the signal level D is estimated from this function, and thefiltering frequency characteristics are controlled on the basis of theestimated amount of noise N, is disclosed in Japanese Patent ApplicationLaid-Open No. 2001-157057. Using this technique, an appropriate noisereduction treatment can be performed on the signal level.

Furthermore, as another example, Japanese Patent Application Laid-OpenNo. 2002-57900 discloses a technique in which the difference value Abetween a pixel of interest and a nearby pixel is determined, then, themean pixel number n used in the moving average method is controlled bythe function n=a/(Δ+b) using the determined difference value Δ andconstant terms a and b that are statically given as constant terms, anda moving average is not determined in cases where the determineddifference value Δ is equal to or greater than a specified thresholdvalue. By using such a technique, it is possible to perform a noisereduction treatment without causing any deterioration of the originalsignal such as edges or the like.

However, since the amount of noise varies dynamically according tofactors such as the temperature at the time of shooting, exposure time,gain and the like, conversion to a function that matches the amount ofnoise during shooting cannot be handled in the case of a technique usingstatic constant terms such as that described in the abovementionedJapanese Patent Application Laid-Open No. 2001-157057, so that theprecision in estimating the amount of noise is inferior. Furthermore,the filtering frequency characteristics are controlled from the amountof noise; however, since this filtering performs equal processingwithout discriminating between flat portions and edge portions, the edgeportions in regions, where it is estimated on the basis of the signallevel that the amount of noise is large, deteriorate. Specifically,processing that discriminates between the original signal and noisecannot be handled, so that the preservation of the original signal ispoor.

Furthermore, in the technique described in Japanese Patent ApplicationLaid-Open No. 2002-57900, the determination of whether or not the movingaverage method is performed is accomplished by comparison with athreshold value. However, since this threshold value is also givenstatically, variation in the amount of noise according to the signallevel cannot be handled, so that the selection of the average number ofpixels or moving average method cannot be optimally controlled.Consequently, noise components remain, resulting in deterioration of theoriginal signal and the like.

SUMMARY OF THE INVENTION

In short, the image pickup system comprises a noise estimating unitwhich estimates the amount of noise contained in the digitized signalfrom an image pickup element in which a plurality of pixels arearranged, either for each pixel or for each specified unit areacomprising a plurality of pixels, and a noise reducing unit whichreduces the noise in the abovementioned signal on the basis of theamount of noise estimated by this noise estimating unit.

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be obvious from thedescription, or may be learned by the practice of the invention. Thefeatures and advantages of the invention may be realized and obtained bymeans of the instrumentalities and combinations particularly pointed outhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram which shows the construction of the imagepickup system in a first embodiment of the present invention;

FIG. 2 is a block diagram which shows the construction of the noiseestimating unit in the abovementioned first embodiment;

FIG. 3A is a diagram which shows an example of the arrangement of the OBregions in the abovementioned first embodiment;

FIG. 3B is another diagram which shows an example of the arrangement ofthe OB regions in the abovementioned first embodiment;

FIG. 4 is a graph which shows the relationship between the variance ofthe OB region and the temperature of the image pickup element in theabovementioned first embodiment;

FIG. 5A is a graph which is used to illustrate the formulization of theamount of noise in the abovementioned first embodiment;

FIG. 5B is another graph which is used to illustrate the formulizationof the amount of noise in the abovementioned first embodiment;

FIG. 6A is a graph which is used to illustrate parameters used in theformulization of the amount of noise in the abovementioned firstembodiment;

FIG. 6B is another graph which is used to illustrate parameters used inthe formulization of the amount of noise in the abovementioned firstembodiment;

FIG. 6C is another graph which is used to illustrate parameters used inthe formulization of the amount of noise in the abovementioned firstembodiment;

FIG. 6D is still another graph which is used to illustrate parametersused in the formulization of the amount of noise in the abovementionedfirst embodiment;

FIG. 7 is a block diagram which shows the construction of the noisereducing unit in the abovementioned first embodiment;

FIG. 8 is a block diagram which shows the construction of the imagepickup system in a second embodiment of the present invention;

FIG. 9 is a diagram which shows the primary color Bayer type filterconstruction in the color filters of the abovementioned secondembodiment;

FIG. 10 is a block diagram which shows the construction of the noiseestimating unit in the abovementioned second embodiment;

FIG. 11 is a block diagram which shows the construction of the noisereducing unit in the abovementioned second embodiment; and

FIG. 12 is a flow chart which shows the noise reduction processing thatis performed by the image processing program in the computer of theabovementioned second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described below withreference to the attached figures.

FIGS. 1 through 7 illustrate a first embodiment of the presentinvention. FIG. 1 is a block diagram which shows the construction of theimage pickup system, FIG. 2 is a block diagram which shows theconstruction of the noise estimating unit, FIGS. 3A and 3B are diagramswhich show examples of the arrangement of the OB regions, FIG. 4 is agraph which shows the relationship between the variance of the OB regionand the temperature of the image pickup element, FIGS. 5A and 5B aregraphs which are used to illustrate the formulization of the amount ofnoise, FIGS. 6A, 6B, 6C and 6D are graphs which are used to illustrateparameters used in the formulization of the amount of noise, and FIG. 7is a block diagram which shows the construction of the noise reducingunit.

As is shown in FIG. 1, this image pickup system comprises a lens system1 which is used to form an image of the subject, an aperture 2 which isarranged inside the lens system 1, and which is used to regulate therange of the luminous flux that passes through the lens system 1, alow-pass filter 3 which is used to eliminate unnecessary high-frequencycomponents from the luminous flux that has been formed into an image bythe abovementioned lens system 1, a CCD 4 constituting a black and whiteimage pickup element which subjects the optical image of the subjectthat is formed via the abovementioned low-pass filter 3 to aphotoelectric conversion, and outputs an electrical image signal, a CDS(correlated double sampling) unit 5 which performs correlateddouble-sampling on the image signal that is output from theabovementioned CCD 4, an amplifier 6 which amplifies the signal that isoutput from the abovementioned CDS unit 5, an A/D converter 7 whichconverts the analog image signal amplified by the abovementionedamplifier 6 into a digital signal, an image buffer 8 which temporarilystores the digital image data that is output from the abovementioned A/Dconverter 7, an exposure control unit 9 which performs light measurementand evaluation relating to the subject on the basis of the image datathat is stored in the abovementioned image buffer 8, and which controlsthe abovementioned aperture 2, CCD 4 and amplifier 6 on the basis of theresults of this evaluation, a focus control unit 10 which detects thefocal point on the basis of the image data stored in the abovementionedimage buffer 8, and which drives an AF motor 11 (described later) on thebasis of the results of this detection, the AF motor 11 which iscontrolled by the abovementioned focus control unit 10, and which drivesthe focusing lens and the like contained in the abovementioned lenssystem 1, a noise estimating unit 13 constituting noise estimating meansthat performs noise estimation (as will be described in detail later) onthe basis of the image data stored in the abovementioned image buffer 8,a noise reducing unit 12 constituting noise reducing means that performsnoise reduction in the image data read out from the abovementioned imagebuffer 8 using the results of the estimation performed by theabovementioned noise estimating unit 13, a signal processing unit 14which processes the image data output from the abovementioned noisereducing unit 12, an output unit 15 which outputs the image data fromthe abovementioned signal processing unit 14 in order to record thisdata on a memory card or the like, an external I/F unit 17 which hasinterfaces to the power supply switch, shutter button, mode switchesthat are used to switch between various shooting modes and the like, anda control unit 16 which constitutes control means comprising amicrocomputer or the like that is connected bidirectionally to theabovementioned CDS unit 5, amplifier 6, A/D converter 7, exposurecontrol unit 9, focus control unit 10, noise reducing unit 12, noiseestimating unit 13, signal processing unit 14, output unit 15, andexternal I/F unit 17, so that this control unit comprehensively controlsthe image pickup system containing these parts, and which alsoconstitutes parameter calculating means and shutter speed calculatingmeans.

Next, the signal flow in an image pickup system such as that shown inFIG. 1 will be described.

The image pickup system is constructed such that the shooting conditionssuch as the ISO sensitivity and the like can be set via the external I/Funit 17. After these settings have been made, the pre-image-pickup modeis entered by half-pressing a shutter button formed by a two-stagepress-button switch.

The video signal that is output by the CCD 4 via the abovementioned lenssystem 1, aperture 2 and low pass filter 3 is read out as an analogsignal by universally known correlated double sampling in the CDS unit5.

The analog signal is amplified by a specified amount by the amplifier 6,converted into a digital signal by the A/D converter 7, and transmittedto the image buffer 8.

The video signal inside the image buffer 8 is subsequently transmittedto the exposure control unit 9 and the focus control unit 10.

The exposure control unit 9 determines the brightness level in theimage, and controls the aperture value of the aperture 2, the electronicshutter speed of the CCD 4, the amplification rate of the amplifier 6and the like with the set ISO sensitivity, shutter speed in the limit ofmovement of the hands and the like being taken into account, so that anappropriate exposure is obtained. Furthermore, the focus control unit 10detects the edge intensity in the image, and obtains a focused image bycontrolling the AF motor 11 such that this edge intensity is maximized.

When preparations for the real shooting have been completed byperforming such a pre-image-pickup mode, the real shooting is thenperformed when it is detected via the external I/F unit 17 that theshutter button has been fully pressed.

The real shooting is performed on the basis of the exposure conditionsdetermined by the exposure control unit 9 and the focusing conditionsdetermined by the focus control unit 10. These shooting conditions aretransmitted to the control unit 16.

When real shooting is thus performed, the video signal is transmitted tothe image buffer 8 and stored in the same manner as in the case ofpre-image-pickup.

The video signal inside the image buffer 8 is transmitted to the noiseestimating unit 13; furthermore, shooting conditions such as theexposure conditions determined by the exposure control unit 9, the ISOsensitivity set by the external I/F unit 17 and the like are alsotransmitted to the noise estimating unit 13 via the control unit 16.

On the basis of the abovementioned information and video signal, thenoise estimating unit 13 calculates the amount of noise in each unit ofa specified size, e.g., in the present embodiment, in each pixel (i.e.,in pixel units), and transmits the calculated amount of noise to thenoise reducing unit 12. The calculation of the amount of noise in thenoise estimating unit 13 is performed under the control of the controlunit 16 in synchronization with the processing of the noise reducingunit 12.

On the basis of the amount of noise calculated by the noise estimatingunit 13, the noise reducing unit 12 performs noise reduction processingon the video signal inside the image buffer 8, and transmits the videosignal that has been subjected to this processing to the signalprocessing unit 14.

Under the control of the control unit 16, the signal processing unit 14performs universally known emphasis processing, compression processingand the like on the video signal that has been subjected to noisereduction processing, and transmits the signal that has been subjectedto this processing to the output unit 15. The output unit 15 records andstores the received video signal on a memory card or the like.

Next, one example of the construction of the noise estimating unit 13will be described with reference to FIG. 2.

The noise estimating unit 13 comprises: an OB region extraction unit 21which extracts the signal of an OB (optical black) region arranged onthe right side of the image region of the CCD 4 (as shown for example inFIG. 3A) from the video signal stored in the image buffer 8 under thecontrol of the control unit 16; a first buffer 22 which stores thesignal of the OB region extracted by the OB region extraction unit 21; avariance calculating unit 23 constituting variance calculating meanswhich reads out the signal of the OB region stored in the abovementionedfirst buffer 22, calculates the variance value of this signal, and usesinformation relating to the exposure conditions transmitted from theabovementioned control unit 16 to perform a correction for the amount ofamplification of the abovementioned amplifier 6 on this variance value;a ROM 25 used for temperature estimation constituting temperatureestimating means in which the pre-measured relationship between thevariance value and the temperature of the image pickup element isrecorded; a temperature estimating unit 24 constituting temperatureestimating means which comprises parameter calculating means thatdetermines the temperature of the image pickup element on the basis ofthe variance value output from the abovementioned variance calculatingunit 23 and information from the abovementioned ROM 25 used fortemperature estimation; a local region extraction unit 26 constitutingsignal value calculating means that extracts local regions of aspecified size in specified positions from the video signal stored inthe abovementioned image buffer 8; a second buffer 27 which stores thesignals of the local regions extracted by the abovementioned localregion extraction unit 26; an average calculating unit 28 constitutingsignal value calculating means which comprises parameter calculatingmeans that reads out the signals of the local regions stored in theabovementioned second buffer 27, calculates the average value of thesesignals, and outputs this average value as the signal value level of thepixel of interest; a gain calculating unit 29 constituting gaincalculating means which comprises parameter calculating means thatcalculates the amount of amplification of the abovementioned amplifier 6on the basis of information relating to the exposure conditions (ISOsensitivity, exposure information, white balance information and thelike) transmitted from the abovementioned control unit 16; a standardvalue assigning unit 30 constituting assigning means that providesstandard values in cases where any of the parameters are omitted; aparameter ROM 32 constituting coefficient calculating means that storeparameters relating to functions (described later) that are used toestimate the amount of noise; a coefficient calculating unit 31constituting coefficient calculating means which comprises noise amountcalculating means that estimates the amount of noise for the pixel ofinterest by means of a specified formula, on the basis of informationrelating to the parameters that are read out from the abovementionedparameter ROM 32, the temperature of the image pickup element that isoutput from the abovementioned temperature estimating unit 24 or theabovementioned standard value assigning unit 30, the signal value levelthat is output from the abovementioned average calculating unit 28 orthe abovementioned standard value assigning unit 30, the amount ofamplification that is output from the abovementioned gain calculatingunit 29 or the abovementioned standard value assigning unit 30, and theshutter speed that is output from the abovementioned control unit 16 orthe abovementioned standard value assigning unit 30; a functioncalculating unit 33 constituting function calculating means whichcomprises noise amount calculating means that calculate the amount ofnoise using functions formulized (as will be described later) by meansof the coefficients output from the abovementioned coefficientcalculating unit 31; and an upper limit setting unit 34 constitutingupper limit value setting means that sets a limit and outputs this limitto the abovementioned noise reducing unit 12 such that the amount ofnoise that is output from the abovementioned function calculating unit33 does not exceed a specified threshold value.

In the present embodiment, since the processing of the noise reducingunit 12 is separated in the horizontal direction and vertical direction(as will be described later), the abovementioned local region extractionunit 26 is devised such that extraction is performed while the overallimage is successively scanned in (for example)4×1 size units in the caseof horizontal-direction processing, and in (for example) 1×4 size unitsin the case of vertical-direction processing. The processing performedby the local region extraction unit 26 is performed in synchronizationwith the processing of the noise reducing unit 12.

Furthermore, the upper limit setting unit 34 is provided inconsideration of cases in which the reduction processing for thetheoretical amount of noise becomes subjectively excessive.Specifically, in cases where the amount of noise is large, there may beinstances in which the original signal is lost and a deterioration inthe image quality is felt subjectively if this noise is completelyeliminated. Accordingly, this unit is devised such that the preservationof the original signal is given priority, and the total image quality isincreased, even if this means that noise components remain. Furthermore,the function of the upper limit setting unit 34 can also be stopped bythe control unit 16 by operation from the external I/F unit 17.Furthermore, the abovementioned control unit 16 is connectedbidirectionally to the abovementioned OB region extraction unit 21,variance calculating unit 23, temperature estimating unit 24, localregion extraction unit 26, average calculating unit 28, gain calculatingunit 29, standard value assigning unit 30, coefficient calculating unit31, function calculating unit 33, and upper limit setting unit 34, andcontrols these units.

The relationship between the variance of the OB region and thetemperature of the image pickup element estimated in the abovementionedtemperature estimating unit 24 will be described with reference to FIG.4.

As is shown in this figure, the temperature of the image pickup elementrises in a monotonic increase while describing a curve as the varianceof the OB region increases.

In the case of random noise in the OB region, in which there is noincident light, dark current noise is the governing factor, and thisdark current noise is related to the temperature of the image pickupelement.

Accordingly, the random noise of the OB region is calculated as avariance value, and the relationship between this variance value and thetemperature variation of the image pickup element is measured beforehandand stored in the ROM 25 used for temperature estimation. As a result,the temperature estimating unit 24 can estimate the temperature of theCCD 4 (which is the image pickup element) from the variance value thatis calculated by the variance calculating unit 23, using thecorresponding relationship that is stored in the ROM 25 used fortemperature estimation.

Furthermore, in the above description, it is considered that thetemperature of the image pickup element is the same at all positions onthe element, and only one temperature is determined. However, thepresent invention need not be limited to this; it would also be possibleto construct this unit such that local temperatures at respective pointson the element are determined.

For example, as is shown in FIG. 3B, it would also be possible to devisethis unit such that OB regions are arranged on the four sides of theimage region, variance values for the OB regions positioned respectivelyon the upper end, lower end, left end and right end are determined withrespect to specified blocks in the image, and variance values for thespecified blocks are determined by a linear interpolation of thesevariance values. As a result, a highly precise temperature estimationcan be accomplished even in cases where the temperature of the imagepickup element is non-uniform.

Next, the formulization of the amount of noise that is used when theamount of noise of the pixel of interest is estimated by the coefficientcalculating unit 31 will be described with reference to FIGS. 5A and 5B.The function of the amount of noise N with respect to the signal valuelevel L is formulized as shown by the following equation (1).

N=AL ^(B) +C  (1)

Here, A, B and C are constant terms, and a constant term is added to afunction expressing a power of the signal value level L.

If the outline of this function in a case where (for example) A>0, 0<B<1and C>0 is plotted, a shape such as that shown in FIG. 5A is obtained.

However, the amount of noise N does not depend on the signal value levelL alone, but also varies according to the temperature of the CCD 4 whichconstitutes the image pickup element, and the gain of the amplifier 6.Accordingly, the example shown in FIG. 5B takes these factors intoaccount.

Specifically, instead of A, B and C which are constant terms in theabovementioned equation (1), a(T, G), b(T, G) and c(T, G) which use thetemperature T and gain G as parameters are introduced as shown inequation (2).

N=a(T,G)L ^(b(T,G)) +c(T,G)  (2)

FIG. 5B shows how the curve indicated by this equation (2) is plotted inthe case of a plurality of gains G (1, 2 and 4 times in the exampleshown in the figure) at a plurality of temperatures T (T1 through T3 inthe example shown in the figure).

In FIG. 5B, an independent variable is expressed as the signal valuelevel L, and a dependent variable is expressed as the amount of noise N.The temperature T which is used as a parameter is plotted as acoordinate axis in the direction perpendicular to these variables.Accordingly, the amount of noise N according to the signal value level Lis read respectively within the plane expressed by T=T1, within theplane expressed by T=T2 and within the plane expressed by T=T3. In thiscase, the variation of the curve shape caused by the gain G (which is aparameter) is expressed by drawing a plurality of curves within therespective planes.

The individual curves indicated by the respective parameters have aconfiguration that is more or less similar to the curve produced byequation (1) as shown in FIG. 5A; however, the respective coefficientsa, b and c naturally vary according to the respective values of thetemperature T and gain G.

FIG. 6A shows an outline of the characteristics of the abovementionedfunction a(T, G), FIG. 6B shows an outline of the abovementionedfunction b(T, G), and FIG. 6C shows an outline of the abovementionedfunction c(T, G).

Since these respective functions are two-variable functions with thetemperature T and gain G as independent variables, FIGS. 6A through 6Care plotted as three-dimensional coordinates, and the plots of thefunctions form curved surfaces in these plotted spaces. Here, however,instead of showing the concrete shapes of the curved surfaces, the majorvariations in the characteristics are shown using curves.

As a result of the temperature T and gain G being input as parametersinto such functions a, b, and c, the respective constants terms A, B andC are output. Furthermore, the concrete shapes of these functions caneasily be acquired beforehand by measuring the characteristics of theimage pickup element system including the CCD 4 and amplifier 6.

Random noise tends to increase as the exposure time becomes longer.Consequently, if the combination of shutter speed and aperture valuediffers, there may be a difference in the amount of noise that isgenerated, even if the amount of exposure is the same. Accordingly, anexample in which a correction is performed with such differences beingtaken into account will be described with reference to FIG. 6D.

Here, a correction coefficient d(S) which uses the shutter speed S as aparameter is introduced, and correction by a formulization of the typeshown in equation (3) is performed by means of multiplying the equation(2) and this correction coefficient together.

N={a(T,G)L ^(b(T,G)) +c(T,G)}d(S)  (3)

The functional shape of this correction coefficient d(S) is obtained bymeasuring the characteristics of the image pickup element systembeforehand. For example, this function has a shape such as that shown inFIG. 6D. FIG. 6D shows the increment D in the amount of noise for theshutter speed S.

As is shown in FIG. 6D, the increment D in the amount of noise has theproperty of increasing abruptly when the shutter speed S is smaller thana certain threshold value S^(TH) (i.e., when the exposure time is long).Accordingly, two different procedures are used according to whether theshutter speed S is higher or lower than this threshold value S^(TH); thefunction d(S) is used in the case of a long exposure time, but theprocedure may be simplified such that a fixed coefficient is used in thecase of a short exposure time.

The four (4) functions a(T, G), b(T, G), c(T, G) and d(S) mentionedabove are stored in the abovementioned parameter ROM 32. Furthermore,the correction for the shutter speed need not always be prepared as afunction, but may also be prepared as some other means, e.g., as a tableor the like. The coefficient calculating unit 31 calculates therespective coefficients A, B, C and D using the four functions stored inthe parameter ROM 32, taking as parameters the temperature T, gain G andshutter speed S that are dynamically acquired (or acquired from thestandard value assigning unit 30).

The function calculating unit 33 determines the functional shapes usedto calculate the amount of noise N by applying the respectivecoefficients A, B, C and D calculated by the abovementioned coefficientcalculating unit 31 to the abovementioned equation (3); this unitcalculates the amount of noise N according to the signal value level Lthat is output from the abovementioned average calculating unit 28 viathe abovementioned coefficient calculating unit 31.

In this case, the respective parameters such as the temperature T, gainG, shutter speed S and the like need not always be determined for eachshooting. For examples, since the temperature T stabilizes after a fixedperiod of time has elapsed following the switching on of the powersupply, it would also be possible for the control unit 16 to storetemperature information calculated in the temperature estimating unit 24following this stabilization in the standard value assigning unit 30,and to omit the subsequent calculation process so that temperatureinformation read out from the standard value assigning unit 30 is used.Thus, in cases where parameters from the temperature estimating unit 24,average calculating unit 28, gain calculating unit 29, control unit 16and the like are not obtained, the standard value assigning unit 30 setsand outputs standard parameters; as a result, the speed of theprocessing can be increased, and power can be saved. Furthermore, thestandard value assigning unit 30 can also output standard values forother required parameters.

Next, one example of the construction of the noise reducing unit 12 willbe described with reference to FIG. 7.

The noise reducing unit 12 comprises: a horizontal line extraction unit41 which successively extracts video signals in units of horizontal linefrom the abovementioned image buffer 8; a first smoothing unit 42constituting smoothing means that scans the image signals of thehorizontal lines extracted by the abovementioned horizontal lineextraction unit 41 in pixel units and perform universally knownhysteresis smoothing with the threshold value from a threshold valuesetting unit 46 (described later) as the amount of noise; a buffer 43which stores the video signal for one screen by successively storing thehorizontal lines that have been smoothed by the abovementioned firstsmoothing unit 42; a vertical line extraction unit 44 which successivelyextracts video signals in units of vertical line from the abovementionedbuffer 43 after a video signal corresponding to one screen has beenaccumulated in the buffer 43; a second smoothing unit 45 constitutingsmoothing means that scans the image signals of the vertical linesextracted by the abovementioned vertical line extraction unit 44 inpixel units, performs universally known hysteresis smoothing with thethreshold value from the threshold value setting unit 46 (describedlater) as the amount of noise, and successively outputs the signals tothe abovementioned signal processing unit 14; and a threshold valuesetting unit 46 constituting threshold value setting means that acquiresthe amount of noise estimated by the abovementioned noise estimatingunit 13 in pixel units in accordance with the horizontal lines extractedby the abovementioned horizontal line extraction unit 41 or the verticallines extracted by the abovementioned vertical line extracting unit 44,sets the amplitude value of the noise as a threshold value (minimumamplitude value), and outputs this value to the abovementioned firstsmoothing unit 42 or the abovementioned second smoothing unit 45.

Here, the hysteresis smoothing performed in the abovementioned first andsecond smoothing units 42 and 45 is performed under the control of thecontrol unit 16 in synchronization with the operation of the noiseestimating unit 13 and the operation of the threshold setting unit 46.

Furthermore, the abovementioned control unit 16 is connectedbidirectionally to the abovementioned horizontal line extraction unit41, vertical line extraction unit 44 and threshold value setting unit46, and controls these units.

Moreover, in the above description, the amount of noise is estimated inpixel units. However, the present invention is not limited to this; itwould also be possible to devise the unit such that the amount of noiseis estimated for each arbitrary specified unit area such as (forexample) 2×2 pixels, 4×4 pixels or the like. In such a case, theprecision of noise estimation drops; on the other hand, this isadvantageous in that higher-speed processing is possible.

In the first embodiment, the amount of noise is estimated for each pixelor each unit area, and noise reduction processing is performed inaccordance with local amounts of noise; accordingly, optical noisereduction is possible in light areas and dark areas, so that ahigher-quality image can be obtained.

Furthermore, the respective parameters relating to the amount of noiseare dynamically determined for each shooting, and the amount of noise iscalculated from these parameters; accordingly, this can be dynamicallyapplied to different conditions for each shooting, so thathigh-precision estimation of the amount of noise is possible.

Furthermore, the amount of noise is set as a threshold value, andsignals that are equal to or less than this threshold value areexcluded; accordingly, signals that exceed the threshold value arepreserved as original signals, so that a high-quality image in whichthere is no deterioration of the edge portions, and in which only noiseis reduced, can be obtained.

Moreover, the calculated amount of noise is limited such that thisamount does not exceed a specified upper-limit value; accordingly,excessive noise reduction processing can be prevented, so thatpreservation of the edge portions of the original signal can be ensured.In this case, the setting or non-setting of such an upper-limit valuecan be accomplished by the operation of the device, so that an approachthat produces a subjectively better image quality can be selected.

In addition, the signal level of the pixel of interest is acquired byaveraging in the region surrounding the pixel of interest; accordingly,the effects of the noise components can be reduced, and high-precisionestimation of the amount of noise is possible.

Furthermore, the temperature of the abovementioned image pickup elementis estimated from the variance value of the OB region of the imagepickup element, and is used as a parameter for the estimation of theamount of noise; accordingly, the amount of noise can be estimated withhigh precision in dynamic response to variations in temperature duringshooting. In this case, since the OB region is utilized, a low-costimage pickup system can be realized.

The amount of gain during shooting is determined on the basis of the ISOsensitivity, exposure information and white balance information, and isused as a parameter in the estimation of the amount of noise;accordingly, the amount of noise can be estimated with high precision indynamic response to variations in the gain during shooting.

The amount of correction for noise is determined in accordance with theshutter speed used; accordingly, the amount of noise can also beestimated with high precision even in the case of noise that increasesduring long-term exposure, in dynamic response to the shutter speedduring shooting.

Standard values are set for parameters that are not obtained duringshooting; furthermore, coefficients for the calculation of the amount ofnoise are determined together with the parameters that are obtained, andthe amount of noise is calculated from these coefficients. Accordingly,the amount of noise can be estimated even in cases where the necessaryparameters are not obtained during shooting, so that a stable noisereducing effect can be obtained. Furthermore, since functions are usedin the calculation of the amount of noise, the required amount of memoryis small, so that costs can be reduced. Moreover, by intentionallyomitting some of the parameter calculations, it is possible to reducecosts and save power.

Thus, in the present embodiment, even if the factors that affect theamount of noise vary dynamically, the amount of noise can beappropriately reduced in accordance with these varying factors, so thata high-quality image can be obtained.

FIGS. 8 through 12 show a second embodiment of the present invention.FIG. 8 is a block diagram which shows the construction of the imagepickup system, FIG. 9 is a diagram which shows the primary color Bayertype filter construction in the color filters, FIG. 10 is a blockdiagram which shows the construction of the noise estimating unit, FIG.11 is a block diagram which shows the construction of the noise reducingunit, and FIG. 12 is a flow chart which shows the noise reductionprocessing that is performed by the image processing program in thecomputer.

In the second embodiment, parts that are the same as in theabovementioned first embodiment are labeled with the same symbols, and adescription of these parts is omitted. For the most part, only thepoints that are different are described.

As is shown in FIG. 8, the image pickup system of the second embodimentcomprises (in addition to the construction of the abovementioned firstembodiment) primary color Bayer type, for example, color filters 51 thatare arranged at the front of the abovementioned CCD 4, a temperaturesensor 52 which is arranged in the vicinity of the abovementioned CCD 4,and which constitutes parameter calculating means used to measure thetemperature of the abovementioned CCD 4 in real time, and to output themeasurement results to the abovementioned control unit 16, a pre-WB unit53 which performs a simple white balance detection on the basis of thevideo signal stored in the abovementioned image buffer 8, and controlsthe abovementioned amplifier 6 on the basis of the detection results,and a color signal separating unit 54 constituting separating means thatread out the video signals stored in the abovementioned image buffer 8,separate the color signals, and output these signals to theabovementioned noise reducing unit 12 and the abovementioned noiseestimating unit 13. The pre-WB unit 53 and color signal separating unit54 are connected to the abovementioned control unit 16 bidirectionally,and are controlled by the control unit 16.

The flow of signals in the image pickup system shown in the FIG. 8 isbasically similar to that in the first embodiment; only the portionsthat are different will be described.

When the pre-image-pickup mode is entered by half-pressing the shutterbutton, the subject image is picked up by the CCD 4 via the colorfilters 51, and the image-captured subject is output as a video signal.

The video signal is subjected to the processing described in the firstembodiment, and is stored in the image buffer 8 as a digital videosignal. The video signal stored in the image buffer 8 is transmitted tothe abovementioned exposure control unit 9 and focus control unit 10,and is also transmitted to the pre-WB unit 53.

The pre-WB unit 53 calculates simple white balance coefficients bymultiplying each color signal by a signal with a specified brightnesslevel in the video signal, and transmits these coefficients to theamplifier 6.

The amplifier 6 performs a white balance adjustment by multiplying eachcolor signal by a different gain using the simple white balancecoefficients received from the pre-WB unit 53.

Next, when it is detected that the shutter button has been fullypressed, a real shooting operation is performed on the basis of theexposure conditions determined by the exposure control unit 9, the focusconditions determined by the focus control unit 10 and the white balancecoefficients determined by the pre-WB unit 53, and these shootingconditions are transmitted to the control unit 16.

The video signal acquired by this real shooting operation is stored inthe image buffer 8, and is then transmitted to the color signalseparating unit 54 and separated into each color of the color filters.

As is described above, the filter construction of the abovementionedcolor filters 51 arranged at the front of the CCD 4 is, for example, aprimary color Bayer type construction as shown in FIG. 9; specifically,the basic unit is 2×2 pixels, with green (G1, G2) filters being arrangedin diagonal positions, and red (R) and blue (B) filters being arrangedin the remaining diagonal positions. Furthermore, the green filters G1and G2 are filters with the same optical characteristics; here, however,these filters are distinguished as G1 and G2 for convenience.

The color signal separating unit 54 is devised such that this unitseparates the video signal inside the image buffer 8 according to thesefour types of color filters R, G1, G2 and B. This separating operationis performed under the control of the control unit 16 in synchronizationwith the processing of the noise reducing unit 12 and the processing ofthe noise estimating unit 13.

The respective color signals separated by the color signal separatingunit 54 are transmitted to the noise estimating unit 13, whereestimation of the amount of noise is performed as described above. Usingthe results of this estimation, noise reduction processing is performedin the noise reducing unit 12, and the respective processed colorsignals are synthesized and transmitted to the signal processing unit14. The subsequent operations are the same as in the abovementionedfirst embodiment.

Next, one example of the construction of the noise estimating unit 13 inthe present embodiment will be described with reference to FIG. 10.

The basic construction of the noise estimating unit 13 is similar tothat of the noise estimating unit shown in FIG. 2 in the abovementionedfirst embodiment; constituent elements that have the same function aredesignated by the same names and labeled with the same symbols.

The noise estimating unit 13 comprises a local region extraction unit 26which extracts a local region of a specified size in a specifiedposition for each color signal output from the abovementioned colorsignal separating unit 54, a buffer 61 which stores the color signals ofthe local regions extracted by the abovementioned local regionextraction unit 26, a gain calculating unit 29 which calculates theamount of amplification of the abovementioned amplifier 6 on the basisof information relating to the exposure conditions and informationrelating to the white balance coefficients transmitted from theabovementioned control unit 16, a standard value assigning unit 30 whichprovides standard values in cases where any of the parameters areomitted, an average and variance calculating unit 62 which reads out thesignals in the abovementioned buffer 61, calculates the average valueand variance value, transmits the calculated average value to thelook-up table unit 63 as the signal value level of the pixel ofinterest, and transmits the variance value to the control unit 16 to useas a control parameter by the noise reducing unit 12, and a look-uptable unit 63 constituting look-up table means which is noise amountcalculating means in which the relationships between the amount of noiseand the shutter speed output from the abovementioned control unit 16 orstandard value assigning unit 30, information relating to thetemperature of the image pickup element output from the abovementionedtemperature sensor 52 or standard value assigning unit 30, amount ofamplification from the abovementioned gain calculating unit 29 orstandard value assigning unit 30, and signal value level output from theabovementioned average and variance calculating unit 62 or standardvalue assigning unit 30 are constructed by means similar to those usedin the abovementioned first embodiment and recorded as a look-up table.

The amount of noise thus obtained by the look-up table 63 is transmittedto the noise reducing unit 12.

Furthermore, the processing of the abovementioned local regionextraction unit 26 is performed in synchronization with the processingof the abovementioned noise reducing unit 12, and the processing of thenoise reducing unit 12 (described later) is performed in block units;accordingly, in the present embodiment, extraction is performed whilethe entire image is successively scanned, for example, in units of 4×4pixels.

Furthermore, the abovementioned control unit 16 is connectedbidirectionally with the abovementioned local region extraction unit 26,average and variance calculating unit 62, gain calculating unit 29,standard value assigning unit 30 and look-up table unit 63, and controlsthese units.

Next, one example of the construction of the noise reducing unit 12 willbe described with reference to FIG. 11.

The noise reducing unit 12 comprises a size setting unit 74 constitutingcontrol value setting means that sets the filter size on the basis ofthe amount of noise estimated by the abovementioned noise estimatingunit 13, a local region extraction unit 71 which extracts pixel blockscorresponding to the filter size set by the abovementioned size settingunit 74 from the respective color signals output from the abovementionedcolor signal separating unit 54 such that these blocks encompass thepixel of interest (e.g., such that these blocks are centered on thepixel of interest), a coefficient ROM 75 constituting smoothing means inwhich coefficients corresponding to a preset filter size are recorded, afiltering unit 72 constituting smoothing means that reads in thecoefficients of the corresponding filter size from the abovementionedcoefficient ROM 75 on the basis of the filter size set by theabovementioned size setting unit 74, and performs filtering processingto apply universally known smoothing to the pixel blocks extracted bythe abovementioned local region extraction unit 71, and a buffer 73which stores the respective color signals subjected to filteringprocessing that are output from the abovementioned filtering unit 72 forall colors such that these signals correspond to the signal outputpositions of the CCD 4.

The abovementioned size setting unit 74 makes a selection from filtersizes of, for example, 1×1 pixels to 9×9 pixels in accordance with theamount of noise estimated by the noise estimating unit 13, such that asmall size is selected when the amount of noise is small, and a largesize is selected when the amount of noise is large. This filter size isa control value for controlling the frequency characteristics of thesmoothing processing; as a result, filtering processing (smoothingprocessing) that reduces a specified frequency band in the video signalis performed in accordance with the frequency characteristics of thenoise.

Furthermore, the abovementioned size setting unit 74 is devised suchthat this unit receives variance value information relating to thesignal value level in the vicinity of the pixel of interest from thecontrol unit 16, discriminates that the pixel of interest constitutes aflat region in cases where this variance value is small, and that thepixel of interest constitutes an edge region in cases where thisvariance value is large, and, on the basis of the discriminationresults, does not perform any correction of the filter size in caseswhere the pixel of interest constitutes a flat region, and corrects thefilter size to a smaller size in cases where the pixel of interestconstitutes an edge region.

Furthermore, filtering processing is performed for all colors byrepeating the abovementioned processing for each of the respective colorsignals, and the respective color signals stored in the buffer 73 aresubsequently read out and processed by the abovementioned signalprocessing unit 14.

Furthermore, the abovementioned control unit 16 is connectedbidirectionally to the abovementioned local region extraction unit 71,filtering unit 72 and size setting unit 74, and controls these units.

In addition, in the above description, it is assumed that processing isperformed by means of hardware. However, the present invention is notlimited to this; processing may also be performed by means of software.

For example, it would also be possible to devise the system such thatthe video signals output from the CCD 4 are taken as raw data in anunprocessed state, and such that information such as the temperatureduring shooting, gain, shutter speed and the like from theabovementioned control unit 16 are added to this raw data as headerinformation. It would also be possible to output the raw data to whichthis header information has been added to a processing device such as acomputer or the like, and to process this data by means of software inthis processing device.

An example in which noise reduction processing is performed by means ofan image processing program in a computer will be described withreference to FIG. 12.

When the processing is started, all color signals constituting the rawdata, as well as header information such as the temperature, gain,shutter speed and the like, are first read in (step S1).

Next, the raw data is separated into respective color signals (step S2),and scanning is performed individually for each color signal (step S3).

Then, local regions of a specified size which are, for example, in unitsof 4×4 pixels centered on the pixel of interest are extracted (step S4).

The average value which is the signal value level of the pixel ofinterest, and the variance value which is used to discriminate between aflat region and an edge region, are calculated for the extracted localregions (step S5).

Next, parameters such as the temperature, gain, shutter speed and thelike are determined from the header information that has been read in.Here, in cases where necessary parameters are not included in the headerinformation, specified standard values are assigned (step S6).

The amount of noise is calculated (step S7) using the look-up table onthe basis of the signal value level calculated in the abovementionedstep S5 and the temperature, gain, shutter speed and the likeconstituting the parameters that are set in the abovementioned step S6.

Next, the filtering size is determined (step S8) on the basis of thevariance value calculated in the abovementioned step S5 and the amountof noise calculated in the abovementioned step S7.

A region corresponding to the filtering size determined in theabovementioned step S8 is then extracted such that this region iscentered on the pixel of interest (step S9).

Next, coefficients corresponding to the filtering size determined in theabovementioned step S8 are read in (step S10).

Smoothing filtering processing is then performed on the region extractedin the abovementioned step S9 using the filtering size determined in theabovementioned step S8 and the coefficients determined in theabovementioned step S10 (step S11).

Then, the smoothed signals are successively read out (step S12), ajudgment is made as to whether or not scanning of the entire signal hasbeen completed for one color (step S13). In cases where this scanninghas not been completed, the processing returns to step S3, and theabovementioned processing is performed until the scanning is completed.

On the other hand, in cases where it is judged that the signal scanninghas been completed in the abovementioned step S13, a further judgment ismade as to whether or not processing has been completed for the colorsignals of all colors (step S14). In cases where processing has stillnot been completed for the color signals of all colors, the processingreturns to step S2, and the abovementioned processing is performed. Onthe other hand, in cases where this processing has been completed, theprocessing is ended.

Furthermore, in the above description, a case in which the color filters51 are primary color Bayer type filters is described as an example.However, the present invention is not limited to this. For example, itwould also be possible to use the present invention in a similar mannerin a case where the color filters are complementary color filters;furthermore, it would also be possible to use the present invention inthe case of a two CCD or three CCD.

In the abovementioned second embodiment, an effect more or less similarto that of the abovementioned first embodiment can be obtained.Furthermore, since signals from an image pickup element that has colorfilters are separated into color signals for each color filter, andsince the amount of noise is estimated for each pixel unit or unit area,and noise reduction processing suited to local amounts of noise isperformed, optimal noise reduction can be performed from light areas todark areas, so that a high-quality image can be obtained. Furthermore,the present invention can be used in various types of image pickupsystems such as primary color or complementary color systems, and singleCCD, two CCD or three CCD systems or the like.

Furthermore, since a filter size corresponding to the amount of noise isselected, and noise reduction processing is performed using this filtersize, only the noise components are removed, and the remaining signal ispreserved as the original signal, so that a high-quality image in whichnoise alone has been reduced can be obtained.

Furthermore, since the temperature of the image pickup element duringshooting is directly measured in real time and used as a parameter forthe estimation of the amount of noise, the amount of noise can beestimated with high precision in dynamic response to temperaturevariations during shooting.

In addition, since standard values are set for parameters not obtainedduring shooting, and the amount of noise is calculated from a look-uptable together with the obtained parameters, the amount of noise can beestimated even in cases where necessary parameters cannot be obtainedduring shooting, so that a stable noise reduction effect can beobtained. Furthermore, since a table is used for the calculation of theamount of noise, high-speed processing is possible. Moreover, areduction in cost and a saving of power can be achieved by intentionallyomitting some of the parameter calculations.

In the present embodiment, even if factors that affect the amount ofnoise in a color CCD vary dynamically, the amount of noise can beappropriately reduced in response to these varying factors, so that ahigh-quality image can be obtained.

Furthermore, the present invention is not limited to the abovementionedembodiments; it goes without saying that various modifications andapplications are possible within limits that involve no departure fromthe spirit of the invention.

As is described above, the image pickup system and image processingprogram of the present invention make it possible to achieve anappropriate reduction of noise in the image, so that a high-qualityimage can be obtained.

In this invention, it is apparent that working modes different in a widerange can be formed on the basis of this invention without departingfrom the spirit and scope of the invention. This invention is notrestricted by any specific embodiment except as may be limited by theappended claims.

1. An image pickup system comprising: a noise estimating unit forestimating the amount of noise contained in digitized signals from animage pickup element in which a plurality of pixels are arranged, foreach pixel or for each specified unit area comprising a plurality ofpixels; a noise reducing unit for reducing the noise contained in thesignals on the basis of the amount of noise estimated by the noiseestimating unit; color filters arranged at the front of the image pickupelement; and a separating unit for separating the signals output fromthe image pickup element into signals for each of the color filters. 2.The image pickup system according to claim 1, wherein the noiseestimating unit comprises: a parameter calculator for calculatingparameters on the basis of at least one type of information selectedfrom among a signal value level of the signals, a temperature of theimage pickup element, a gain for the signals, and a shutter speed duringshooting; and a noise amount calculator for calculating the estimatedamount of noise on the basis of the parameters calculated by theparameter calculator.
 3. The image pickup system according to claim 1,wherein the noise estimating unit comprises an upper limit value settingunit for setting an upper limit value on the estimated amount of noise.4. The image pickup system according to claim 1, wherein the noisereducing unit comprises: a threshold value setting unit for setting anamplitude value of the noise as a threshold value for each pixel or foreach specified unit area comprising a plurality of pixels on the basisof the amount of noise estimated by the noise estimating unit; and asmoothing unit for reducing the amplitude component in the signals whichare below the threshold value set by the threshold value setting unit.5. The image pickup system according to claim 1, wherein the noisereducing unit comprises: a control value setting unit for settingcontrol values used to control the frequency characteristics of thesmoothing processing on the basis of the amount of noise estimated bythe noise estimating unit; and a smoothing unit for performing smoothingprocessing that reduces a specified frequency band in the signals on thebasis of the control values set by the control value setting unit. 6.The image pickup system according to claim 2, wherein the parametercalculator comprises a signal value calculator for determining thesignal value levels by averaging a plurality of pixel values in one of anearby region of a specified size and in the unit area that includes thepixel of interest.
 7. The image pickup system according to claim 2,wherein the parameter calculator comprises a temperature sensor thatmeasures a temperature of the image pickup element.
 8. The image pickupsystem according to claim 2, wherein the image pickup element comprisesan optical black (OB) region, and the parameter calculator comprises: avariance calculator for calculating the variance of the signals in theOB region; and a temperature estimating unit for estimating thetemperature of the image pickup element based on the variance calculatedby the variance calculator.
 9. The image pickup system according toclaim 2, wherein the parameter calculator comprises a gain calculatorfor determining the gain based on at least one type of informationselected from among ISO sensitivity, exposure information and whitebalance information.
 10. The image pickup system according to claim 2,wherein the parameter calculator comprises a shutter speed calculatorfor determining the shutter speed during the shooting based on exposureinformation.
 11. The image pickup system according to claim 2, whereinthe noise amount calculator calculates the amount of noise N using asignal value level L of the signals, a temperature T of the image pickupelement, a gain G for the signals and a shutter speed S during shootingas parameters, and the noise amount calculator comprises: a coefficientcalculator for calculating four (4) coefficients A, B, C and D on thebasis of three (3) functions a(T, G), b(T, G) and c(T, G) using thetemperature T and gain G as parameters, and a function d(S) using theshutter speed S as a parameter; and a function calculator forcalculating the amount of noise N based on a functional equationN=(ALB+C)D defined by the four (4) coefficients A, B, C and D calculatedby the coefficient calculator.
 12. The image pickup system according toclaim 11, wherein the noise amount calculator further comprises anassigning unit for assigning standard parameter values, and theparameters are values calculated by one of the parameter calculator, andstandard values assigned by the assigning unit.
 13. The image pickupsystem according to claim 2, wherein the noise amount calculatorcomprises: an assigning unit for assigning standard parameter values forparameters not obtained from the parameter calculator; and a look-uptable for determining the amount of noise by inputting the signal valuelevel, temperature, gain and shutter speed obtained from one of theparameter calculator and the assigning unit.
 14. An image pickup systemcomprising: a noise estimating unit for estimating the amount of noisecontained in digitized signals from an image pickup element in which aplurality of pixels are arranged, for each pixel or for each specifiedunit area comprising a plurality of pixels; a noise reducing unit forreducing the noise contained in the signals on the basis of the amountof noise estimated by the noise estimating unit; the noise reducing unitcomprising: a control value setting unit for setting control values usedto control the frequency characteristics of the smoothing processing onthe basis of the amount of noise estimated by the noise estimating unit;and a smoothing unit configured to reduce a specified frequency band inthe signals based on the control values set by the control value settingunit.
 15. An image pickup system comprising: a noise estimating unit forestimating an amount of noise contained in digitized signals from animage pickup element in which a plurality of pixels are arranged, forone of each pixel and each specified unit area comprising a plurality ofpixels; a noise reducing unit for reducing the noise contained in thesignals on the basis of the amount of noise estimated by the noiseestimating unit; the noise estimating unit comprising: a parametercalculator for calculating parameters based on at least one type ofinformation selected from a signal value level of the signals, atemperature of the image pickup element, a gain for the signals and ashutter speed during shooting; and a noise amount calculator forcalculating the estimated amount of noise based on the parameterscalculated by the parameter calculator; the parameter calculatorcomprising a temperature sensor that measures the temperature of theimage pickup element.
 16. An image pickup system comprising: a noiseestimating unit for estimating an amount of noise contained in digitizedsignals from an image pickup element in which a plurality of pixels arearranged, for one of each pixel and each specified unit area comprisinga plurality of pixels; a noise reducing unit for reducing the noisecontained in the signals on the basis of the amount of noise estimatedby the noise estimating unit; the noise estimating unit comprising: aparameter calculator for calculating parameters based on at least onetype of information selected from a signal value level of the signals, atemperature of the image pickup element, a gain for the signals and ashutter speed during shooting; and a noise amount calculator forcalculating the estimated amount of noise based on the parameterscalculated by the parameter calculator; the noise amount calculatorcomprising: an assigning unit for assigning standard parameter valuesfor parameters not obtained from the parameter calculator; and a look-uptable for determining the amount of noise by inputting the signal valuelevel, temperature, gain and shutter speed obtained from one of theparameter calculator and the assigning unit.
 17. An image pickup systemcomprising: a separating unit for separating digitized signals from animage pickup element which has one of primary- and complementary-colorfilters arranged at a front thereof into color signals for each of thecolor filters; a signal value calculator for determining a signal valuelevel for the respective color signals by averaging a plurality of pixelvalues in one of a nearby region of a specified size and a unit areathat includes a pixel of interest; a gain calculator for determining again for the signals based on at least one type of information selectedfrom ISO sensitivity, exposure information and white balanceinformation; a look-up table for determining the amount of noisecorresponding to the signal value level and the gain for the respectivecolor signals; a small amplitude value setting unit for setting a smallamplitude value for one of each pixel and each specified unit areacomprising a plurality of pixels based on the amount of noise for therespective color signals; and a smoothing unit for removing amplitudecomponents that are equal to or less than the small amplitude value setby the small amplitude value setting unit for the respective colorsignals.
 18. An image processing method, comprising: obtaining an imageof an object employing an image pickup element having an array of pixelsfor generating digitized signals of the image; estimating an amount ofnoise contained in the digitized signals from one of each pixel and atleast one specified unit area comprising a plurality of pixels; settingcontrol values based on the estimated amount of noise; and reducing aspecified frequency band in the signals based on the control values tothereby obtain a high-quality image.
 19. An image processing method,comprising: obtaining an image of an object employing an image pickupelement having one of primary- and complementary-color filters arrangedbetween the object and the pickup element; separating digitized signalsof the image generated by the image pickup element into color signalsfor each of the color filters; determining a signal value level for therespective color signals by averaging a plurality of pixel values in oneof a nearby region of a specified size and a unit area that includes apixel of interest; determining a gain for the color signals based on atleast one type of information selected from ISO sensitivity, exposureinformation and white balance information; determining an amount ofnoise based on the signal value level and the gain for the respectivecolor signals: assigning a small amplitude value to one of each pixeland a specified unit area comprising a plurality of pixels based on theamount of noise for each of the respective color signals; and excludingsignals whose amplitudes are equal to or less than the small amplitudevalue assigned to one of each pixel and the specified unit area for therespective color signals to obtain a high-quality image.